The Role of Fixation Position in Detecting Scene Changes across Saccades

Target objects presented within color images of naturalistic scenes were deleted or rotated during a saccade to or from the target object or to a control region of the scene. Despite instructions to memorize the details of the scenes and to monitor for object changes, viewers frequently failed to notice the changes. However, the failure to detect change was mediated by three other important factors: First, accuracy generally increased as the distance between the changing region and the fixation immediately before or after the change decreased. Second, changes were sometimes initially missed, but subsequently noticed when the changed region was later refixated. Third, when an object disappeared from a scene, detection of that disappearance was greatly improved when the deletion occurred during the saccade toward that object. These results suggest that fixation position and saccade direction play an important role in determining whether changes will be detected. It appears that more information can be retained across views than has been suggested by previous studies. Subjective experience leads viewers to believe that their visual system delivers a complete and veridical representation of the scene before them—a representation akin to a relatively detailed color photograph. This phenomenology forms the basis for the majority of theoretical work in both human and machine vision. A constraint on human perception, though, is that high-acuity vision is restricted to a small foveal region surrounding the current fixation point, with acuity dropping off precipitously from that focal point (Riggs, 1965). The visual system handles this constraint by rapidly reorienting the eyes an average of three times each second via saccadic eye movements (Buswell, 1935; Henderson & Hollingworth, 1999; Rayner, 1998; Yarbus, 1967). Construction of a complete visual representation would therefore seem to require the storage of a high-resolution image across saccades, with images from consecutive fixations overlapped or spatially aligned to form the composite image (Breitmeyer, Kropfl, & Julesz, 1982; Davidson, Fox, & Dick, 1973; Duhamel, Colby, & Goldberg, 1992; Jonides, Irwin, & Yantis, 1982; McConkie & Rayner, 1976). According to this hypothesis, changes to the viewed scene from one fixation to the next should be highly detectable. Recent studies show, however, that human observers often fail to notice seemingly salient changes when the changes occur during a saccade (Grimes, 1996; Henderson, 1997; McConkie, 1990; McConkie & Currie, 1996). Similar change blindness is found when a change occurs to a scene during a brief blanking period (Rensink, O’Regan, & Clark, 1997) or a film cut (Levin & Simons, 1997), or even across the interposition of an opaque object in the real world (Simons & Levin, 1998). The phenomenon of change blindness is strikingly counterintuitive and theoretically important, first because it undermines the traditional view that the visual system constructs a complete representation of the external world, and second because it calls into question the assumption that conscious perceptual experience directly reflects the underlying visual representation. Instead, it appears that humans perceive a complete and detailed world despite the fact that the underlying visual representation is abstract and incomplete (Dennett, 1991). The conditions under which change blindness can be observed are not clear, however. Specifically, previous studies provided no direct evidence that participants were fixating, or had ever fixated, the changing region before or during the change. For example, in the saccade-contingent scene-change studies that have been reported (Grimes, 1996; McConkie, 1990; McConkie & Currie, 1996), the image change was generated during the nth ordinal saccade, where n was predefined prior to the experiment. Thus, the position of the fixation before or following the nth saccade was not used to constrain n. Acuity functions provide evidence that changes might have been missed simply because the changed region appeared only in the visual periphery. Similarly, eye movements typically have not been monitored in paradigms that have shown poor change detection across other sorts of blank periods (but see Hollingworth & Henderson, 1998, for an exception). Studies that have monitored eye movements during free scene viewing have shown that scene detail is preferentially encoded at fixation (Henderson, Weeks, & Hollingworth, 1999; Nelson & Loftus, 1980; Parker, 1978). In the present study, we introduced a new methodology (object changes contingent on a saccade to or from a predefined critical scene region) to investigate the sensitivity of the visual system to scene changes when fixation position relative to the changing region is precisely controlled. In our paradigm, computer-rendered color images of naturalistic scenes were changed contingent on a saccade toward or away from a prespecified target object (see Fig. 1). In the toward condition, the change took place during the first saccade that brought the eyes to the target object; in the away condition, the change took place during the saccade that took the eyes away from the target object immediately after it had been fixated the first time. We also included a control condition in which the target object changed during the first saccade to a nontarget object that was present elsewhere in the scene. To investigate the nature of the information that is encoded and retained across a saccade, two types of object changes were compared. In the rotation condition, the target object instantaneously rotated 90o around its vertical axis during the saccade. In the deletion condition, the target object instantaneously disappeared from the scene during the saccade. Finally, no-change catch trials were included to provide an assessment of the false alarm rate in the experiment. Participants were instructed to view each of 35 scenes in preparation for a later memory test, and to press a response button if and when they noticed a change to any of the objects in the currently viewed scene. If change detection is independent of fixation position, as has been assumed implicitly in the change-blindness literature, then detection rates would be equivalent in the three saccade conditions (toward, away, and control). If instead, as previous eye movement studies suggest, information is preferentially encoded from fixated objects in a Address correspondence to John M. Henderson, Department of Psychology, 129 Psychology Research Building, Michigan State University, East Lansing, MI 48824-1117; e-mail: [email protected]. PSYCHOLOGICAL SCIENCE John M. Henderson and Andrew Hollingworth VOL. 10, NO. 5, SEPTEMBER 1999 439 Fig. 1. Sample scene before (a) and after (b) the object change in a rotation trial. In (a), the boundary region defined around the target object is marked in blue. A change was made to the scene either during the first saccade entering this region (toward condition) or the first saccade exiting this region (away condition). The picture in (b) shows the scan pattern of 1 viewer in the toward condition. Dots represent fixations and lines connecting dots represent saccades. Eye movements before the change are marked in light green and after the change are marked in red. This participant did not detect the change. PSYCHOLOGICAL SCIENCE Detecting Changes Across Saccades scene, then detection would be better when the eyes fixated an object prior to its change (away condition) than when they did not (toward and control conditions). This hypothesis also predicted a detection gradient, with detection rate declining as a function of the distance of the nearest fixation to the changed object. Finally, according to the saccade-target theory of visual stability (Irwin, McConkie, CarlsonRadvansky, & Currie, 1994; McConkie & Currie, 1996), an object that is about to be fixated is preferentially encoded, retained, and compared across a saccade. Thus, this theory predicted that participants would be particularly sensitive to scene changes in the toward condition.